Information Gain Feature Selection for Ordinal Text Classification using Probability Re-distribution

This paper looks at feature selection for ordinal text classification. Typical applications are sentiment and opinion classification, where classes have relationships based on an ordinal scale. We show that standard feature selection using Information Gain (IG) fails to identify discriminatory features, particularly when they are distributed over multiple ordinal classes. This is because inter-class similarity, implicit in the ordinal scale, is not exploited during feature selection. The Probability Re-distribution Procedure (PRP), introduced in this paper, explicates inter-class similarity by revising feature distributions. It aims to influence feature selection by improving the ranking of features that are distributed over similar classes, relative to those distributed over dissimilar classes. Evaluations on three datasets illustrate that the PRP helps select features that result in significant improvements on classifier performance. Future work will focus on automated acquisition of inter-class similarity knowledge, with the aim of generalising the PRP for a wider class of problems.